Under-determined linear systems and $\ell_q$-optimization thresholds

Recent studies of under-determined linear systems of equations with sparse solutions showed a great practical and theoretical efficiency of a particular technique called l1-optimization. Seminal works [7, 20] rigorously confirmed it for the first time. Namely, [7, 20] showed, in a statistical context, that l1 technique can recover sparse solutions of under-determined systems even when the sparsity is linearly proportional to the dimension of the system. A followup [13] then precisely characterized such a linearity through a geometric approach and a series of work [40,41,43] reaffirmed statements of [13] through a purely probabilistic approach. A theoretically interesting alternative to l1 is a more general version called lq (with an essentially arbitrary q). While l1 is typically considered as a first available convex relaxation of sparsity norm l0, lq, 0 ≤ q ≤ 1, albeit non-convex, should technically be a tighter relaxation of l0. Even though developing polynomial (or close to be polynomial) algorithms for non-convex problems is still in its initial phases one may wonder what would be the limits of an lq, 0 ≤ q ≤ 1, relaxation even if at some point one can develop algorithms that could handle its non-convexity. A collection of answers to this and a few realted questions is precisely what we present in this paper. Namely, we look at the lq-optimization and how it fares when used for solving under-determined linear systems with sparse solutions. Although our results are designed to be only on an introductory/conceptual level, they already hint that lq can in fact provide a better performance than l1 and that designing the algorithms that would be able to handle it in a reasonable (if not polynomial) time is certainly worth further exploration.